Classification with rejection: concepts and formal evaluations

Standard classification allocates all processed elements to given classes. Such type of classification assumes that there are only native and no foreign elements, i.e. all processed elements are included in given classes. The quality of standard classification can be measured by two factors: numbers of correctly and incorrectly classified elements, called True Positives and False Positives. Admitting foreign elements in standard classification increases False Positives and, in this way, deteriorates quality of classification. In this context, it is desired to reject foreign elements, i.e. to not assign them to any of given classes. Rejecting foreign elements will reduce the number of False Positives, but can also reject native elements reducing True Positives as side effect. Therefore, it is important to build well designed rejection, which will reject significant part of foreigners and only few natives. In this paper, concepts of evaluations of classification with rejection are presented. Three main models: a classification without rejection, a classification with rejection, and a classification with reclassification are presented. The concepts are illustrated by flexible ensembles of binary classifiers with theoretical evaluations of each model. The proposed models can be used, in particular, as classifiers working with noised data, where recognized input is not limited to elements of known classes.